Touchpad based devices typically suffer from a common problem. When a stylus or similar device is used to draw a line or write some text on the touch screen of a touchpad device, there is a noticeable time lag between the actual pen movement and the “digital ink” appearing on screen. FIG. 1 is an exemplary illustration of “pen-to-ink” latency on a touchpad device. Stylus 120 is used to draw a line on touch screen 122 of device 112. As shown on touch screen 122, there is a time lag between the current position 126 of the stylus and the digital ink 125 being shown on the screen as it updates in response to the user's motion.
This pen-to-ink latency is caused by a combination of factors, including hardware, software and operating system (OS) related factors and varies from device to device. FIGS. 2A and 2B illustrate how, for example, pen-to-ink latency can be quantified as a result of the way in which the touchpad device processes user-initiated events in connection with displaying frames of display information. Specifically, FIG. 2A illustrates how a conventional touchpad device processes frames of pixel information. FIG. 2B illustrates how user initiated events are processed to be displayed on the screen of a conventional touchpad device.
A conventional device, e.g., an Android touchpad device running the JellyBean operating system, may typically refresh screen content at a 60 Hz frequency. Accordingly, the screen is refreshed 60 times each second and each update takes around 16.66 ms (1000 ms/60=16.66 ms). In other words, there are 60 display updates within a second and, as a result, the device has around 16 ms to prepare each frame before it gets presented to the user. As shown in FIG. 2A, each frame, e.g. Frame 0 202 and Frame 1 204, is 16.66 ms long and gets refreshed and replaced by a fresh frame at the v-sync 202 boundary at the end of every 16.66 ms interval.
In a conventional touchpad device, during the frame preparation time all touch events are collected and delivered to an application, e.g., a drawing application, on the next v-sync boundary. For example a user touch event 250 during Frame 0 212 will not be delivered to the application until v-sync 254. Assuming the application uses single buffering, this results in a minimum 16 ms delay before results of the user interaction are actually presented on display. However, most operating systems on conventional devices, e.g. JellyBean, use triple buffering which make the delay even larger.
Multiple buffering comprises the use of more than one buffer to hold a block of data so that a reader will see a complete version of the data rather than a partially updated version of the data being created by a writer. Triple buffering is a technique that uses three buffers to reduce or avoid the problems single buffering can cause. Typically, the front buffer holds the image currently being displayed. The middle buffer holds a completed image until the front buffer is ready to display it. When the current frame finishes displaying, the front buffer and middle buffer switch. The back buffer holds the image being drawn. When the graphics processing unit (GPU) finishes drawing, the back buffer and the middle buffer switch. Using three buffers decouples the drawing process from the display process. This allows a consistent high frame rate to be achieved using v-sync, even if frame drawing takes longer than the frame display time.
FIG. 2B illustrates how triple buffering increases the perceived pen-to-ink latency on screen. During Frame 0 212, for example, the application is rendering to the Frame Buffer 0 (FB0) and at v-sync 254, it displays the contents of Frame Buffer 2 (FB2). During Frame 1 214, the application is rendering to FB2 and displays the contents of Frame Buffer 1 (FB1) at v-sync 255. The touch event 250 that gets delivered at v-sync 254 gets rendered into FB2 during Frame 1 214. Thus, by virtue of the operating system framework, there is already a default touch-delivery latency (TDL) 280 introduced before a touch event is even delivered and processed due to the sample rate of the touch screen itself. In other words, the TDL latency is due to the sample frequency of the touch screen itself and is different from the latency added as a result of the display refresh rate. During Frame 2 216, the frame buffers cycle again and render to FB1 while displaying the contents of FB0 at v-sync 256. Finally, during Frame 3 218, the application renders to FB0 and displays the contents of FB2 at v-sync 257.
The touch event 250 that was rendered into FB2 at the beginning of Frame 1 214 finally gets displayed on screen at v-sync 257. Accordingly, the perceived pen-to-ink latency in a conventional touchpad device with triple buffering is between 48 and 64 ms (3×16 ms+TDL {0-16 ms depending on when the touch event occurred during the frame}). Assuming an average of 8 ms for the TDL, typical applications on conventional touchpad devices have a 56 ms perceived pen-to-ink latency.
Finally, additional latency that may need to be accounted for is latency between the physical movement of the stylus and touch event 250 delivery to the operating system caused by hardware and the driver. Accordingly, the total latency may be higher than 64 ms.
High latency results in lack of responsiveness and may lead to poor user experience, which can ultimately be a decisive factor in a consumer's decision whether to purchase a certain device. Further, prior ways of trying to reduce latency related issues are unsatisfactory. For example, the latency can be improved by using single or double buffering instead of triple buffering because it reduces the number of buffers that need to be cycled through before an event is displayed on screen. However, this is unsatisfactory because the advantages of triple buffering, e.g., a high frame rate are lost.